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Abstract 



We present a simple and clear foundation for finite inference that unites and significantly extends 
the approaches of Kolmogorov and Cox. Our approach is based on quantifying lattices of logical 
■ statements in a way that satisfies general lattice symmetries. With other applications such as 

measure theory in mind, our derivations assume minimal symmetries, relying on neither negation nor 
continuity nor differentiability. Each relevant symmetry corresponds to an axiom of quantification, 
and these axioms are used to derive a unique set of quantifying rules that form the familiar probability 
Qh ' calculus. We also derive a unique quantification of divergence, entropy and information. 

2 ! 1 Introduction 



The quality of an axiom rests on it being both convincing for the application(s) in mind, and compelling 
in that its denial would be intolerable. 
^ ■ We present elementary symmetries as convincing and compelling axioms, initially for measure, 

subsequently for probability, and finally for information and entropy. Our aim is to provide a simple 

■ and widely comprehensible foundation for the standard quantification of inference. We make minimal 
assumptions — not just for aesthetic economy of hypotheses, but because simpler foundations have 
wider scope. 

It is a remarkable fact that algebraic symmetries can imply a unique calculus of quantification. 
Section [5] gives the background and outlines the procedure and major results. Section [3] lists the 
symmetries that are actually needed to derive the results, and the following Section|2]writes each required 
symmetry as an axiom of quantification. In Section [5j we derive the sum rule for valuation from the 
associative symmetry of ordered combination. This sum rule is the basis of measure theory. It is usually 

■ taken as axiomatic, but in fact it is derived from compelling symmetry, which explains its wide utility. 
^ , There is also a direct-product rule for independent measures, again derived from associativity. Section [B] 

derives from the direct-product rule a unique quantitative divergence from source measure to destination. 

In Section [7] we derive the chain product rule for probability from the associativity of chained order 
(in inference, implication). Probability calculus is then complete. Finally, Section [8] derives the Shannon 
entropy and information (a.k.a. Kullback-Leibler) as special cases of divergence of measures. All these 
formulas are uniquely defined by elementary symmetries alone. 

Our approach is constructivist, and we avoid unnecessary formality that might unduly confine our 
readership. Sets and quantities are deliberately finite since it is methodologically proper to axiomatize 
finite systems before any optional passage towards infinity. R.T. Cox 1^ showed the way by deriving the 
unique laws of probability from logical systems having a mere three elementary "atomic" propositions. 
By extension, those same laws applied to Boolean systems with arbitrarily many atoms and ultimately, 
where appropriate, to well-defined infinite limits. However, Cox needed to assume continuity and 
differentiability to define the calculus to infinite precision. Instead, we use arbitrarily many atoms 
to define the calculus to arbitrarily fine precision. Avoiding infinity in this way yields results that cover 
all practical applications, while avoiding unobservable subtleties. 



1 



Our approach unites and significantly extends the set-based approach of Kolmogorov [5] and the 
logic-based approach of Cox T, to form a foundation for inference that yields not just probability 
calculus, but also the unique quantification of divergence and information. 



2 Setting the Scene 

We model the world (or some interesting aspect of it) as being in a particular state out of a finite set 
of mutually exclusive states (as in Figure 1, left). Since we and our tools are finite, a finite set of states, 
albeit possibly very large in number, sufiices for all practical modeling. 

As applied to inference, each state of the world is associated, via isomorphism, with a statement 
about the world. This results in a set of mutually exclusive statements, which we call atoms. Atoms 
are combined through logical OR to form compound statements comprising the elements of a Boolean 
lattice (Figure 1, right), which is isomorphic to a Boolean lattice of sets (Figure 1, center). Although 
carrying different interpretations, the mathematical structures are identical. Set inclusion "c" is 
equivalent to logical implication , which we abstract to lattice order "<" . It is a matter of choice 
whether to include the null set 0, equivalent to the logical absurdity _L. The set-based view is ontological 
in character and associated with Kolmogorov, while the logic-based view is epistemological in character 
and associated with Cox. 



Figure 1. The Boolean lattice of potential states (center) is constructed by taking the 2^ 
powerset of an antichain of N mutually exclusive atoms (in this case ai, 02, 03, left). This lattice 
is isomorphic to the Boolean lattice of logical statements ordered by logical implication (right) . 
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Quantification proceeds by assigning a real number to(x) — x, called a valuation, to elements x. 
(Typewriter font denotes lattice elements x, whereas their associated valuations (real numbers) x are 
shown in italic.) We require valuations to be faithful to the lattice, in the sense that 

< =^ ^x < y ^ (1) 

lattice elements real numbers 



so that compound elements carry greater value than any of their components. Clearly, this by itself is 
only a weak restriction on the behavior of valuation. 

Combination of two atoms (or disjoint compounds) into their compound is written as the operator 
U, for example z = x U y. Our first step is to quantify the combination of disjoint elements through an 
operator © that combines values (Table 1 below lists such operators and their eventual identifications) . 

z — X ®y representing z = x U y (2) 

real numbers joined elements 

We find that the symmetries underlying U place constraints on that effectively require it to be 
addition -t-. At this stage, we already have the foundation of measure theory, and the generalization 
of combination (of disjoint elements) to the lattice join (of arbitrary elements) is straightforward. The 
wide applicability of these underlying symmetries explains the wide utility of measure theory, which 
might otherwise be mysterious. 



2 



We can consider the atoms ai, a2, as, . . . , ajv and bi,b2,...,bM from separate problems as NM 
composite atoms Cij — ai x hj in an equivalent composite problem. The direct- product operator (X) 
quantifies the composition of values: 

c = g (g) 6 representing c a x b (3) 

real numbers composite element 

We find that the symmetries of x place constraints on (g) that require it to be multiplication. 

It is common in science to acquire numerical assignments by optimizing a variational potential. By 
requiring consistency with the numerical assignments of ordinary multiplication, we find that there is 
a unique variational potential H{p \ q), of "plogp" form, known as the (generalized KuUback-Leibler) 
Bregman divergence of measure p from measure q. 

Inference involves the relationship of one logical statement (predicate x) to another (context t), 
initially in a situation where x ^ t so that the context includes subsidiary predicates. To quantify 
inference, we assign real numbers p(x | t), ultimately recognised as probability, to predicate-context 
intervals [x,t]. Such intervals can be chained (concatenated) so that [x,z] = [[x, y], [y, z]], with 
representing the chaining of values. 

p(x I z) = p(x I y) p(y I z) representing [x, z] [[x, y], [y, z]] (4) 

V ' " V ' 

real numbers chained intervals 

We find that the symmetries of chaining require to be multiplication, yielding the product rule of 
probability calculus. When applied to probabilities, the divergence formula reduces to the information, 
also known as the KuUback-Leibler formula, with entropy being a variant. 



2.1 The Order-Theoretic Perspective 

The approach we employ can be described in terms of order-preserving (monotonic) maps between order- 
theoretic structures. Here we present our approach, described above, from this different perspective. 

Order-theoretically, a finite set of exclusive states can be represented as an antichain, illustrated 
in Figure l(left) as three states oi, 02, and situated side- by-side. Our state of knowledge about the 
world (more precisely, of our model of it — we make no ontological claim) is often incomplete so that we 
can at best say that the world is in one of a set of potential states, which is a subset of the set of all 
possible states. In the case of total ignorance, the set of potential states includes all possible states. In 
contrast, perfect knowledge about our model is represented by singleton sets consisting of a single state. 
We refer to the singleton sets as atoms, and note that they are exclusive in the sense that no two can 
be true. 

The space of all possible sets of potential states is given by the partially-ordered set obtained from 
the powerset of the set of states ordered by set inclusion. For an antichain of mutually exclusive 
states, the powerset is a Boolean lattice (Figure 1, center), with the bottom element optional. By 
conceiving of a statement about our model of the world in terms of a set of potential states, we have an 
order-isomorphism from the Boolean lattice of potential states ordered by set inclusion to the Boolean 
lattice of statements ordered by logical implication (Figure 1, right). This isomorphism maps each set of 
potential states to a statement, while mapping the algebraic operations of set union U and set intersection 
n to the logical OR and AND, respectively. 

The perspective provided by order theory enables us to focus abstractly on the structure of a Boolean 
lattice with its generic algebraic operations join V and meet A. This immediately broadens the scope 
from Boolean to more general distributive lattices — the first fruit of our minimalist approach. For 
additional details on partially ordered sets and lattices in particular, we refer the interested reader to 
the classic text by Birkhoff [3] or the more recent text by Davey & Priestley [1]. 

Quantification proceeds by assigning valuations to(x) = x to elements x, to form a real-valued 
representation. For this to be faithful, we require an order-preserving (monotonic) map between the 
partial order of a distributive lattice and the total order of the chains that are to be found within. Thus 
X < y is to imply that x < y, a relationship that we call fidelity. The converse is not true: the total 
order imposed by quantification must be consistent with but can extend the partial order of the lattice 
structure. 

We write the combination of two atoms into a compound element (and more generally any two 
disjoint compounds into a compound element) as U, for example z = x U y. Derivation of the calculus 
of quantification starts with this disjoint combination operator, where we find that its symmetries place 
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constraints on its representation © that allow us the convention of ordinary addition "® — +". This 
basic result generalizes to the standard join lattice operator V for elements that (possibly having atoms 
in common) need not be disjoint, for which the sum rule generalizes to its standard inclusion/exclusion 
form [5], which involves the meet A for any atoms in common. 

There are two mathematical conventions concerning the handling the nothing-is-true null clement _L at 
the bottom of the lattice known as the absurdity. Some mathematicians opt to include the bottom element 
on aesthetic grounds, whereas others opt to exclude it because of its paradoxical interpretation [3]. If it 
is included, its quantification is zero. Either way, fidelity ensures that other elements are quantified by 
positive values that are positive (or, by elementary generalization, zero). At this stage, we already have 
the foundation of measure theory. 

Logical deduction is traditionally based on a Boolean lattice and proceeds "upwards" along a chain 
(as in the arrows sketched in Figure 1). Given some statement x, one can deduce that x implies xORy 
since xORy includes x. Similarly, xANDy implies x since x includes xANDy. The ordering relationships 
among the elements of the lattice are encoded by the zeta function of the lattice [B] 



Deduction is definitive. 

Inference, or logical induction, is the inverse of deduction and proceeds "downwards" along a 
chain, losing logical certainty as knowledge fragments. Our aim is to quantify this loss of certainty, in 
the expectation of deriving probability calculus. This requires generalization of the binary zeta function 
C(x, y) to some real-valued function p{x \ y) which will turn out to be the standard probability of x GIVEN 
y. However, a firm foundation for inference must be devoid of a choice of arbitrary generalizations. By 
viewing quantification in terms of an order-preserving map between the partial order (Boolean lattice) 
and a total order (chain) subject to compelling symmetries alone, we obtain a firm foundation for 
inference, devoid of further assumptions of questionable merit. 

By considering atoms (singleton sets, which are the join-irreducible elements of the Boolean 
lattice) as precise statements about exclusive states, and composite lattice elements (sets of several 
exclusive states) as less precise statements involving a degree of ignorance, the two perspectives of 
logic and sets, on which the Cox and Kolmogorov foundations are based, become united within the 
order-theoretic framework. 

In summary, the powerset comprises the hypothesis space of all possible statements that one can 
make about a particular model of the world. Quantification of join using -I- is the sum rule of probability 
calculus, and is required by adherence to the symmetries we list. It fixes the valuations assigned to 
composite elements in terms of valuations assigned to the atoms. Those latter valuations assigned to 
the atoms remain free, unconstrained by the calculus. That freedom allows the calculus to apply to 
inference in general, with the mathematically-arbitrary atom valuations being guided by insight into a 
particular application. 

2.2 Commentary 

Our results — the sum rule and divergence for measures, and the sum and product rules with information 
for probabilities — are standard and well known (their uniqueness perhaps less so). The matter we 
address here is which assumptions are necessary and which are not. A Boolean lattice, after all, is a 
special structure with special properties. Insofar as fewer properties are needed, we gain generality. 
Wider applicability may be of little value to those who focus solely on inference. Yet, by showing that 
the basic foundations of inference have wider scope, we can thereby offer extra — and simpler — guidance 
to the scientific community at large. 

Even within inference, distributive problems may have relationships between their atoms such that not 
all combinations of states are allowed. Rather than extend a distributive lattice to Boolean by padding 
it with zeros, the tighter framework immediately empowers us to work with the original problem in its 
own right. Scientific problems (say, the propagation of particles, or the generation of proteins) are often 
heavily conditional, and it could well be inappropriate or confusing to go to a full Boolean lattice when 
a sparser structure is a more natural model. 

We also confirm that commutativity is not a necessary assumption. Rather, commutativity of measure 
is imposed by the associativity and order required of a scalar representation. Conversely, systems that are 
not commutative (matrices under multiplication, for example) cannot be both associative and ordered. 



zeta function : 




(5) 
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3 Symmetries 



Here, we list the relevant symmetries on which our axioms are based. All are properties of distributive 

lattices, and our descriptions arc styled that way so that a reader wary of further generality does not 
need to move beyond this particular, and important, example. However, one may note that not all the 
properties of a distributive lattice (such as commutativity of the join) are listed, which implies that these 
results are applicable to a broader class of algebraic structures that includes distributive lattices. 
Valuation assignments rank statements via an order-preserving map which we call fidelity . 



Symmetry : 



lattice elements 



< y ^ 

real numbers 



(6) 



It is a matter of convention that we choose to order the valuations in the same sense as the lattice order 
("more is bigger"). Reverse order would be admissible and logically equivalent, though less convenient. 

In the specific case of Boolean lattices of logical statements, the binary ordering relation, represented 
generically by <, is equivalent to logical implication (=>) between different statements, or equivalently, 
proper subset inclusion (c) in the powerset representation. Combination preserves order from the right 
and from the left 



Symmetry 1 : 



X < y 



X U z < y U z 
z U X < z U y 



(7) 



for any z (a property that can be viewed as distributivity of U over <) on the grounds that ordering 

needs to be robust if it is to be useful. 

Combination is also taken to be associative 



Symmetry 2 : 



(x U y) U z = X U (y U z) 



(8) 



Independent systems can be considered together (Figure 2). 

Figure 2. One system might, for example, be playing-card suits x e ^, A, while another 
independent system might be music keys t € {t», The direct-product combines the spaces 
ofx and t to form the joint space ofx x t with atoms like ^ x tl- 




The direct-product operator x is taken to be (right-)distributive over U 



Symmetry 3 : 



(x X t) U (y X t) = (x U y) X t 



(9) 



so that relationships in one set, such as perhaps J|t U "s?, remain intact whether or not an independent 

element from the other, such as perhaps t], is appended. Left distributivity may well hold but is not 
needed. The direct product of independent lattices is also taken to be associative (Figure 3). 



Symmetry 4 : 



(u X v) X w = u X (v X w) 



(10) 
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Figure 3. Associativity of direct product can be viewed geometrically. 
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Finally, we consider a totally ordered set of logical statements that form a chain x < y < z < t. 
We focus on an interval on the chain, which is defined by an ordered pair of logical statements [x,t]. 
Adjacent intervals can be chained, as in [[x, y], [y, z]] = [x, z], and chaining is associative 



[x,y],[y,z] , [z,t] = [x,y], [y,z],[z,t] 
Using Greek symbols to represent an interval, a — [x, y], — [y, z], 7 = [z, t], we have 



(11) 



Symmetry 5 : (a, ^), 7 = a, (/3, 7) (12) 

These and these alone are the symmetries we need for the axioms of quantification. They are presented 
as a cartoon in the "Conclusions" section below. 

4 Axioms 

We now introduce a layer of quantification. Our axioms arise from the requirement that any quantification 
must be consistent with the symmetries indicated above. Therefore, each symmetry gives rise to an 
axiom. We seek scalar valuations to be assigned to elements of a lattice, while conforming to the above 
symmetries (#0 — ^5)for disjoint elements. 

Fidelity (symmetry ^0) requires us to choose an increasing measure so that, without loss of generality, 
we may set m(_L) = and thereafter 

Axiom : a; > (13) 

To conform to the ordering symmetry ^1, we require © as set up in Ec^uation [5] to obey 

^ ^ \ x®z<y®z 

Axiom 1 : x < v s ^ ^ 

^ \ z(Sx < z®y (.-^4^ 

To conform to the associative symmetry #2, we also require ® to obey 



Axiom 2 : {x ® y) ® z = x ® (y ® z) (15) 

These equations are to hold for arbitrary values x, y, z assigned to the disjoint x, y, z. Appendix A will 
show that these order and associativity axioms are necessary and sufficient to determine the additive 
calculus of measure. 

To conform to the distributive symmetry ^3, we require ® as set up in Equation [3] to obey 



Axiom 3 : [x®t)<S>{y®t)^(x®y)®t (16) 

for disjoint x and y combined with any t from the second lattice. Presence of t may change the measures, 
but does not change their underlying additivity. To conform to the associative symmetry ^^4, we also 
require (g) to obey 



6 



Axiom 4 : 



(u (g) (g) w = u (Xi (w (8) w) 



(17) 



These axioms determine the muhiphcative form of (g) and also lead to a unique divergence between 
measures. 

To conform to the associative symmetry #5, we require as set up in Equation |4] to obey 



where a = [x, y], /3 = [y, z], 7 = [z,t] are individual steps concatenated along the chain a,j3,^, which 
is [[x, y], [y, z], [z,t]] = [x,t]. This final axiom will let us pass from measure to probability and Bayes' 
theorem, and from divergence to information and entropy. For each operator (Table 1), the eventual 
form satisfies all relevant axioms, which assures existence. Uniqueness remains to be demonstrated. 

5 Measure 

Preliminary to investigating probability, we attend to the foundation of measure. 
5.1 Disjoint arguments 

According to the scalar associativity theorem (Appendix A), an operator obeying axioms 1 and 2 exists 
and can without loss of generality be taken to be addition +, giving the sum rule. 

Sum rule : x ® y = x + y 

(19) 

Commutativity x (B y = y (B x, though not explicitly assumed, is an unsurprising property. In 
accordance with fidelity (axiom 0), element values are strictly positive re > 0. In this form, positive- valued 
valuation m(x) = a; of lattice elements is known as a measure. If the null element is included as the 
bottom of the lattice, it has zero value. 

Whilst we are free to adopt additivity as a convenient convention, we are also free to adopt any 
order-preserving regrade Q for which the rule would be 



This carries no extra generality because this form can be reverted to additivity by applying 0, but we 
need such alternative grading later to avoid inconsistency between different assignments. There is no 
other freedom. If the linear form of sum rule is to be maintained, the only freedom is linear rescaling 
Q{x) = Kx, with > to retain positivity. 

Measure theory (see for example [7]) is usually introduced with additivity (countably additive or 
cr-additive) and non-negativity as "obvious" basic assumptions, with emphasis on the technical control 
of infinity in unbounded applications. Here we emphasize the foundation, and discover the reason why 
measure theory is constructed as it is. The symmetries of combination require it. Any other formulation 
would break these basic properties of associativity and order, and would not yield a widely useful theory. 

5.2 Arbitrary Arguments 

For elements x and y that need not be disjoint, their join V is defined as comprising all their constituent 
atoms counted once only, and the meet A as comprising those atoms they have in common. In inference, 
V is logical OR and A is logical AND. 

By putting x = u U v and y = v U w for disjoint u, v,w, we reach the general "inclusion/exclusion" 
sum rule for arbitrary x and y 



Axiom 5 : 




(18) 




(20) 



m{x V y) + m(x A y) 



m(x) + m{y) 



(21) 



Commutativity of join and meet follow: 



m(x V y) 



r7i(yVx), m(x A y) 



m(y A x) . 



(22) 
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5.3 Independence 

From the associativity of direct product (axiom 4), the associativity theorem (Appendix A again) assures 
the existence of an additivity relationship of the form 

e{x(git) ^ e{x) + e{t) (23) 

for some invertible function Q of the measures x — m(x), t =^ m(t) and x <^ t — m(x x t). We can not 
proceed as before to re-grade in terms of Q{m) to supersede m, because we are aheady using additivity 

x(»t + y(»t={x + y)®t (24) 

(axiom 3, distributivity of (E) over ©=+) to define the grade. Instead, we require consistency with the 
sum- rule behavior for x (E)t and y (E)t. Defining ^I" = gives, term by term, 

*(e + T) + *(?] + r) = *(C(e,??) +r) (25) 

where 



^ = e(x), v = Q{y), C^&{x + y), T = e(t). (26) 

Among these variables, ^,r],T are independent, but (through the sum rule), C depends on ^ and 77 but 
not T. This is the product equation. By definition, returns a measure, so it is positive. 

The product theorem (Appendix B) shows Q to be logarithmic, with Equation 1231 reading 

1, x®t 1, X 1, t 

^log^ = ^log- + -^log- (27) 

with A and C universal constants {A cancelling out), and C being positive. The obvious convention 
C — I loses no generality, and shows ig) to be simple multiplication 



Direct-product rule : x®t = xt (28) 

Measures are required to multiply, because of associativity of direct product, and the operation is 

represented by "scale by i" . This is consistent with linear rescaling (here depending on the second factor 
t) being the only allowed freedom for the measure assigned to the first factor x. 



6 Variation 

Variational principles are common in science — minimum energy for equilibrium, Hamilton's principle 
for dynamics, maximum entropy for thermodynamics, and so on — and we seek one for measures. The 
aim is to discover a variational potential H{m) whose constrained minimum allows the valuations m = 
(mi, 7712, . . . , mAf) of N atoms to be assigned subject to appropriate constraints of the form /(m) — 
constant. (The vectors which appear in this section are shown in bold-face font.) 

The variational potential is required to be general, applying to arbitrary constraints. Just like values 
themselves, constraints on individual atom values can be combined into compound constraints that 
influence several values: indeed the constraints could simply be imposition of definitive values. Such 
combination allows a Boolean lattice, entirely analogous to Figure 1, to be developed from individual 
atomic constraints. The variational potential H is to be a valuation on the measures resulting from these 
constraints, combination being represented by some operator O so that 



H{xmiY{y)=H{x)OH{y) (29) 

for constraints acting on disjoint atoms or compounds. 

Adding extra constraints always increases iJ, otherwise the variational requirement would be broken, 
so H must be faithful to chaining in the lattice. 

=^ H{x) < H{y) (30) 



real numbers 

We also have order 
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H{x)<H{y) =^ [ H{z)OH{x)<H{z)OH{y) ^^^> 

because if y is a "harder" constraint than x (meaning H{y) > H{x)), that ranking should not be affected 
by some other constraint on something else. Associativity 

(^H{x) O (y)) O H{z) = H{x) O (H{y) O i?(z)) (32) 

is likewise required and expresses the combination of three constraints. It would also be natural to assume 
commutativity, H{x)OH{y) — H{y)OH{x), but that is not necessary because we already recognize 
Equations [501 [53 as our axioms 0, 1, 2. Hence, using Appendix A again, there exists a "O = +" grade 
on which H is additive. 

i?(m)= J2 ^'(™') (33) 

atoms i 

We have now justified additivity, thus filling a gap in traditional accounts of the calculus of variations. 
Under perturbation, the minimization requirement is 

6H{m)>0 when (5/i(m) = (5/2(m) = • • • = (34) 

The standard "© = +" form of the sum rule happens to be continuous and differentiable, so is applicable 
to valuation of systems that differ arbitrarily little. We adopt it, and can then justifiably require the 
variational potential to be valid for arbitrarily small perturbations: 

dH{m) = when d/i(m) = d/2(m) = • • • = (35) 

This limit Equation |35] is weaker than the original Equation |34] not only because of the restricted context, 
but also because the nature of the extremum (maximum or minimum or saddle) is lost in the discarded 
second-order effects. However, it still needs to be satisfied. It also shows that any variational potential 
must by its nature be differentiable at least once. 

One now invents supposedly constant "Lagrange multiplier" coefficients Ai, A2, . . . and considers what 
appears at first to be the different problem of solving 

d ( _ff(m) — Ai/i(m) — A2/2(m) — ...)= under arbitrary perturbation (36) 

for m. Clearly, Equation [551 is equivalent to Equation [35] for perturbations that happen to hold the /'s 
constant {df = 0). However, the values those /'s take may well be wrong. The trick is to choose the A's 
so that the /'s take their correct constraint values. That being done, Equation solves the variational 
problem Equation [35] 

Let the application be two-dimensional, x-hy-y, in the sense of applying to values to(xx y) of elements 
on a direct-product lattice. Suppose we have x-dependent constraints that yield m(x) — nix on one factor 
(say the card suits in Figure 2 above), and similar y-dependent constraints that yield m{y) = my on 
the other factor (say music keys in Figure 2). Both factors being thus controlled, their direct-product is 
implicitly controlled by the those same constraints. Here, we already know the target value m(x x y) = 
rrixmy from the direct-product rule Equation 1281 Hence the variational assignment for the particular 
value to(x x y) derives from 

H'^yim^niy) = Ai/i(toj;) + \2f2{my) (37) 

(where ' indicates derivative). The variational theorem (Appendix C) gives the solution of this functional 
equation as 

Hi{mi) ^ Ai + BiUii + Ciirui logm^ - rui) (38) 

for the individual valuation being considered, where Ai,Bi,Ci are constants. Combining all the 
atoms yields 



H{m) = ^ (^Aj + B^rrii + Q {m, log mi 



atoms i 



(39) 
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The coefficient Ci represents the intrinsic importance of atom in the summation, but usually the 
atoms are a priori equivalent so that the C's take a common value. The scaling of a variational potential 
is arbitrary (and is absorbed in the Lagrange multipliers), so we may set C = 1, ensuring that H has 
a minimum rather than a maximum. Alternatively, C = — 1 would ensure a maximum. However, the 
settings of A and B depend on the application. 

6.1 Divergence and Distance 

One use of _ff is as a quantifier of the divergence of destination values w from source values u that 
existed before the constraints that led to w were applied. For this, we set C = 1 to get a minimum, 
Bi = — log Mi to place the unconstrained minimizing w at u, and Ai — Ui to make the minimum value 
zero. This form is 

Divergence : iJ(w | u) — (ui — Wi + Wi \og{wi / Ui)) 

atoms i 

(40) 

This formula is unique: none other has the properties Equations I33I37I that elementary applications 
require. Equivalently, any different formula would give unjustifiable answers in those applications. 
Plausibly, H is non-negative, H{w | u) > with equality if and only if w = u, so that it usefully 
quantifies the separation of destination from source. 

In general, H obeys neither commutativity nor the triangle inequality, H{w \ u) ^ H{u \ w) and 
H{w I u) ^ H{w I v) + H{v I u). Hence it cannot be a geometrical "distance", which is required to 
have both those properties. In fact, there is no definition of geometrical measure-to-measure distance 
that obeys the basic symmetries, because H is the only candidate, and it fails. 

Here again we see our methodology yielding clear insight. "From-to" can be usefully quantified, but 
"between" cannot. A space of measures may have connectedness, continuity, even differentiability, but 
it cannot become a metric space and remain consistent with its foundation. 

In the limit of many small values, H admits a continuum limit 

H{w I u) = y (uiO) - w{9) + w{9) \og{w{9)/u{e))) d9 (41) 

The constraints that force a measure away from the original source may admit several destinations, but 
minimizing H is the unique rule that defines a defensibly optimal choice. This is the rationale behind 
maximum entropy data analysis ^15) . 

7 Probability Calculus 

In inference, we seek to impose on the hypothesis space a quantified degree of implication p(x \ t), to 
represent the plausibility of predicate x conditional on current knowledge that excludes all hypotheses 
outside the stated context t. This is accomplished via a bivaluation, which is a functional that takes a 
pair of lattice elements to a real number. This bivaluation should depend on both x (obviously) and t 
(otherwise it would be just the measure assigned to x). The natural conjecture is that probability should 
be identified with a normalized measure, and we proceed to prove this — measures can have arbitrary 
total but probabilities will (according to standard convention) sum to unity. 

At the outset, though, we simply wish to set up a bivaluation for predicate x within context t. 

7.1 Chained Arguments 

Within given context t, we require p(x | t) to have the order and associative symmetries =ffl and ^2 
that define a measure. Consequently, p obeys the sum rule 

p(xUy |t)=p(x|t)+p(y|t) (42) 

for disjoint x and y with x U y < t. It is the dependence on t that remains to be determined. 
Associativity of chaining (axiom 5) for a < b < c < d is represented by 

(^pia^QpijD^'^Qpi^^ (43) 

p{a) p{l3) p(7) p(q) p{l3) p(7) 

p(a,/3) p(^,7) 
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We do not have commutativity, (a, /3) — [[a, b], [b, c]] ~ [a, c] not being the same as (/3,a) (which is 
meaningless), but we do have associativity and we do have order along the chain. By the associativity 
theorem, exists and there is a scale on which it is simple addition. However, we can not regradc to that 
scale and discard the original because we have already fixed the grade of -p to be additive with respect 
to its first argument. Instead, we infer additivity on some other grade 0(p) 

e( p(aj^ ) -e(p(aj^) +e(p(bj_c)) (44) 

p(a)0p(/3) p(/3) 

required to be consistent with the sum-rule behavior of p. Defining = gives 

p(aj_c) = vl/(e(p(aj^) + e(p(bj_c^)) (45) 

p(a)0p(/3) p(q) p(/3) 

Substituting this in the sum rule Equation I42[ term by term, yields the same "product Equation" [^5] 

*(C(e,'7)+T) = vI/(e + T) + vI/(r, + r) (46) 

as before, where 

e = e(p(x|z)), 77 = e(Ky|z)), c = e(p(xuy |z)), T = e(p(z|t)). (47) 

Through the sum rule, depends as shown on ^ and r/ but not r. The independent variables are ^,ri,T. 
The solution (Appendix B again) shows Q to be logarithm, so that was multiplication and 

p(x|z)-p(x|y)p(y|z)/C (48) 

in which p (positive by virtue of being a measure on predicates) takes the sign of a universal constant 
C. Without loss of generality, we assign the scale of p by fixing C = 1, giving the standard product rule 
for conditioning. 



Chain-product rule : p{x \ z) — p{x \ y)p{y \ z) (49) 

7.2 Arbitrary Arguments 

The chain-product rule, which as written above is valid for any chain, can be generalized to accommodate 
arbitrary elements. This is accomplished by noting that x A y = x in a chain where x < y, so that 
p{x A y I y) = p(x | y). The general form 



p(aAb I c) =p(a I bA c)p(b I c) (50) 

follows by observing that x = aAbAc, y = bAc and z ~ c form a chain and hence are subject to the 
chain rule. 

The special case p(t | t) = 1 is obtained by setting y = z = t in the chain-product rule. For any 
X < t, ordering requires p{x \ t) < p{t | t) = 1, so that the range of values is < p < 1 and we recognize 
p as probability , hereafter denoted Pr. 

Probability calculus is now proved: 



Range 
Sum rule 
Chain-product 



= Pr(_L I t) < Pr(x | t) < Pr(t | t) = 1 
Pr(x V y I t) + Pr(x A y | t) = Pr(x | t) + Pr(y | t) 
Pr(xAy I t) =Pr(x | yAt) Pr(y | t) 



The top element of the current lattice, t, is the (provisional) truth, often written T. 

From the commutativity Pr(x A y | t) = Pr(y A x | t) associated with A, we obtain Bayes' Theorem 



Pr(x I 6* A t) Pr(6l | t) = Pr(6l | x A t) Pr(x | t) 
which can be simplified by making the common context implicit and writing 



(51) 
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Pr(x I e) Pr(0) = Pr(6l | x) Pr(x) || t (52) 

Likelihood Prior Posterior Evidence 

to relate data x and parameter 9 (with context t understood). Do not misinterpret the abbreviated 
notation. Probability is always and necessarily, by construction, a bivaluation that assigns a real number 
to a pair of elements in a Boolean lattice. In addition, one does not need to differentiate between 
likelihood, prior, posterior, and evidence by giving each one a different notation. The terms that comprise 
Bayes' Theorem represent the same bivaluation applied to different pairs of elements. 

7.3 Probability as a Ratio 

The equations of probability calculus (range, sum rule, and chain-product rule) can all be subsumed in 
the single expression 

Pr(x I t) = a^^^pil Vx, Vt ^ ± (53) 
m(t) 

for probability as a ratio of measures. Thus the calculus of probability is nothing more than the 
elementary calculus of proportions of measure. As anticipated, within its context t, a probability 
distribution is simply the shape of the confined measure, automatically normalized to unit mass. 

This is, essentially, the original discredited frequentist definition (see [8]) of probability, as the ratio of 
number of successes to number of trials. However, it is here retrieved at an abstract level, which bypasses 
the catastrophic difficulties of literal frequentism when faced with isolated non-reproducible situations. 
Just as ordinary addition is forced for measures in [0, oo), so ordinary proportions in [0, 1] are forced for 
probability calculus. 

Whereas the sum rule for measure and probability generalizes to the inclusion/exclusion form for 
general elements which need not be disjoint, so does the ratio form of probability allow generalization 
from intervals [3] to generalized intervals, consisting of arbitrary pairs [x,t] which need not be in a 
chain. The bivaluation form Equation[53]still holds but now represents a general degree of implication 
between arbitrary elements. 

8 Information and Entropy 

Here, we take special cases of the variational potential H , appropriate for probability distributions instead 
of arbitrary measures. 

8.1 Information 

Within a given context, probability is a measure, normalized to unit mass. The divergence H of 
destination probability p from sourc e probability q then simplif ies to 



Information : 



H{p I q) = ^pfclog — 

k ^'^ 



(54) 



In statistics, this is known as the Kullback-Leibler formula [S]. 

If the final destination is a fully determined state, with a single p equal to 1 while all the others are 
necessarily 0, then we have the extreme case 

^(P I q) = - log^fc when Pk = 1 ■ (55) 

This represents the information gained on acquiring the knowledge that the specific k was 
true — equivalently the surprise at finding k instead of any available alternative. Generally, H is the 
amount of compression (logarithmically, with respect to the source) induced by the constraints that 
modulate source into destination. 

In the limit of many small values, H admits a continuum limit 

H{p\ci)= fp{x)\ogP^dx (56) 

J q{x) 

sometimes (with a minus sign) known as the cross-entropy. 
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8.2 Entropy 



The variational potential 



(57) 



k 



can also quantify uncertainty. For this, we require zero uncertainty when one probability value equals to 
1 (definitely present) and all the others are necessarily (definitely not present). This is accomplished 
by setting Ak — and Bk = C. Setting C = — 1 gives the conventional scale, and yields 



We call this "entropy" , and give it a separate symbol S as well as a separate name, to distinguish it from 
the previous "information" special case of divergence. 

Entropy happens to be the expectation value of the information gained by deciding on one particular 
cell instead of any of the others in a partition. 



It is a function of the partitioning as well as the probability distribution, which is why it does not have 
a continuum limit. Plausibly, entropy has the following three properties: 

• 5* is a continuous function of its arguments. 

• If there are n equal choices, so that pk — ^/n, then S is monotonically increasing in n. 

• If a choice is broken down into subsidiary choices, then 
S adds according to probabilistic expectation, meaning 

S{pi,P2,P3) = S{pi,P2+P3) + {P2+P3)S{p2,P3)- 

These are the three properties from which Shannon |10) originally proved the entropy formula. Here, we 
see that those properties, like that formula, are inevitable consequences of seeking a variational quantity 
for probabilities. 

Information and entropy are near synonyms, and are often used interchangeably. As seen here, 
though, entropy S is different from H. It is a property of just one partitioned probability distribution, 
it has a maximum not a minimum, and it does not have a continuum limit. Its least value, attained 
when a single probability is 1 and all the others are 0, is zero. Its value generally diverges upwards as 
the partitioning deepens, whereas H usually tends towards a continuum limit. 

9 Conclusions 
9.1 Summary 

We start with a set {ai, a2, aa, . . . , aAr} of "atomic" elements which in inference represent the most 
fundamental exclusive statements we can make about the states (of our model) of the world. Atoms 
combine to form a Boolean lattice which in inference is called the hypothesis space of statements. This 
structure has rich symmetry, but other applications may have less and we have selected only what we 
needed, so that our results apply more widely and to distributive lattices in particular. The minimal 
assumptions are so simple that they can be drawn as the cartoon below (Figure 4). 

Axiom 1 represents the order property that is required of the combination operator U. Axiom 2 says 
that valuation must conform to the associativity of U. These axioms are compelling in inference. By 
the associativity theorem (Appendix A — see the latter part for a proof of minimality) they require the 
valuation to be a measure to(x), with U represented by addition (the sum rule). Any 1:1 regrading is 
allowed, but such change alters no content so that the standard linearity can be adopted by convention. 
This is the rationale behind measure theory. 

The direct product operator x that represents independence is distributive (axiom 3) and associative 
(axiom 4), and consequently independent measures multiply (the direct-product rule). There is then a 
unique form of variational potential for assigning measures under constraints, yielding a unique divergence 
of one measure from another. 

Probability Pr(a; | t) is to be a bivaluation, automatically a measure over predicate x within any 
specified context t. Axiom 5 expresses associativity of ordering relations (in inference, implications) and 



Entropy : 




(58) 



^(p) = ( - logp,). 



(59) 
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leads to the chain-product rule which completes probability calculus. The variational potential defines 
the information (Kullback-Leibler) carried by a destination probability relative to its source, and also 
yields the Shannon entropy of a partitioned probability distribution. 

9.2 Commentary 

We have presented a foundation for inference that unites and significantly extends the approaches of 
Kolmogorov [2] and Cox [1], yielding not just probability calculus, but also the unique quantification of 
divergence and information. Our approach is based on quantifying finite lattices of logical statements 
in such a way that quantification satisfies minimal required symmetries. This generalizes algebraic 
implication, or equivalently subset inclusion, to a calculus of degrees of implication. It is remarkable 
that the calculus is unique. 

Our derivations have relied on a set of explicit axioms based on simple symmetries. In particular, we 
have made no use of negation (NOT), which in applications other than inference may well not be present. 
Neither have we assumed any additive or multiplicative behavior (as did Kolmogorov [2] , de Finetti [TT] , 
and Dupre & Tipler |12|). On the contrary, we find that sum and product rules follow from elementary 
symmetry alone. 

We find that associativity and order provide minimal assumptions that are convincing and compelling 
for scalar additivity in all its applications. Associativity alone does not force additivity, but associativity 
with order does. Positivity was not assumed, though it holds for all applications in this paper. 

Commutativity was not assumed either, though commutativity of the resulting measure follows as a 
property of additivity. Associativity and commutativity do not quite force additivity because they allow 
degenerate solutions such as a© 6 = max(a, b). To eliminate these, strict order is required in some form, 
and if order is assumed then commutativity does not need to be. Hence scalar additivity rests on ordered 
sequences rather than the disordered sets for which commutativity would be axiomatic. 

Associativity + Order Additivity allowed Commutativity 

Associativity alone Additivity allowed 

Associativity + Commutativity Additivity allowed 

Aczel |13) assumes order in the form of reducibility, and he too derives commutativity. However, his 
analysis assumes the continuum limit already attained, which requires him to assume continuity. 

Associativity + Order + Continuity Additivity allowed Commutativity 

Our constructivist approach uses a finite environment in which continuity does not apply, and proceeds 
directly to additivity. Here, continuity and differentiability are merely emergent properties of + as the 
continuum limit is approached by allowing arbitrarily many atoms of different type. 

Yet there can be no requirement of continuity, which is merely a convenient convention. For example, 
re-grading could take the binary representations of standard arguments (IOI.OII2 representing 5|) and 
interpret them in base-3 ternary (with IOI.OII3 representing 10^), so that O(IO^) = 5|. Valuation 
becomes discontinuous everywhere, but the sum rule still works, albeit less conveniently. Indeed, no 
finite system can ever demonstrate the infinitesimal discrimination that defines continuity, so continuity 
cannot possibly be a requirement of practical inference. 

At the cost of lengthening the proofs in the appendices, we have avoided assuming continuity or 
differentiability. Yet we remark that such infinitesimal properties ought not infiuence the calculus of 
inference. If they did, those infinitesimal properties would thereby have observable effects. But detecting 
whether or not a system is continuous at the infinitesimal scale would require infinite information, 
which is never available. So assuming continuity and differentiability, had that been demanded by the 
technicalities of mathematical proof (or by our own professional inadequacy), would in our view have 
been harmless. As it happens, each appendix touches on continuity, but the arguments are appropriately 
constructed to avoid the assumption, so any potential controversy over infinite sets and the role of the 
continuum disappears. 

Other than reversible regrading, any deviation from the standard formulas must inevitably contradict 
the elementary symmetries that underlie them, so that popular but weaker justifications (e.g., 
de Finetti |llj ) in terms of decisions, loss functions, or monetary exchange can be discarded as 
unnecessary. In fact, the logic is the other way round: such applications must be cast in terms of 
the unique calculus of measure and probability if they are to be quantified rationally. Indeed, we hold 
generally that it is a tactical error to buttress a strong argument (like symmetry) with a weak argument 
(like betting, say). Doing that merely encourages a skeptic to sow confusion by negating the weak 
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argument, thereby casting doubt on the main thesis through an illogical impression that the strong 
argument might have been circumvented too. 

Finally, the approach from basic symmetry is productive. Goyal and ourselves [14] have used just 
that approach to show why quantum theory is forced to use complex arithmetic. Long a deep mystery, 
the sum and product rules of complex arithmetic are now seen as inevitably necessary to describe the 
basic interactions of physics. Elementary symmetry thus brings measure, probability, information and 
fundamental physics together in a remarkably unified synergy. 
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A Appendix A: Associativity Theorem 



Atoms X, y, z,. . . , or disjoint lattice elements more generally, are to be assigned valuations x,y,z, 

If valuations coincide (thoiigh other marks may differ), such atoms arc said to be of the same type. We 
allow arbitrarily many atoms of arbitrarily many types. Our proof is constructive, with combinations 

built as sequences of atoms appended one at a time, x U y U . . . having valuation x (B y (B The 

consequent stand-alone derivation is rather long, but avoids making what would in our finite environment 
be an unnatural assumption of continuity. We also avoid assuming that an inverse to combination exists. 
We merely assume order (axiom 1) 

Axiom la: (x(Bz<y(Bz 
Axiom lb: ^ V \ z®x < z®y 

and associativity (axiom 2) 

Axiom 2 : {x ® y) ® z = x ® {y ® z) 

Theorem: 



Axiom 1 (order) and axiom 2 (associativity) imply that 

X (B y = Q-^ (&{x) + e{y)'^ 
for any order-preserving regrade Q of "© = +" applied to scalar values. 



A.l Proof: 

The form quoted in the theorem is easily seen to satisfy both axioms 1 and 2, which demonstrates 
existence of a calculus ® of quantification. The remaining question is whether this calculus is unique. 

We start by building sequences from just one type of atom before introducing successively more 
types to reach the general case. In this way, we lay down successively finer grids. Whenever another 
atom is introduced to generate a new sequence, that new sequence's value inevitably lies somewhere at, 
between, or beyond previously assigned values. If it lies within an interval, we are free to choose it to be 
anywhere convenient. Such choice loses no generality, because the original value could be recovered by 
order-preserving regrade of the assignments. Values can be freely and reversibly regraded in and only in 
any way that preserves their order. Any such mapping preserves axiom 1, but reversal of ordering would 
allow the axiom to be broken. 

Most points of the continuum escape this approach and arc never accessed, so we do not allow 
ourselves continuum properties such as continuity. We build our finite system from the bottom up, using 
only those values that we actually need. 

By interchanging x and y in axiom 1, the same relationship holds when "<" is replaced throughout 
by ">" , and replacement by "=" holds trivially. So, in effect, the axiom makes a three-fold assertion 

a;| = jy a;ez| = jye^; and ^©.t| = | z y (60) 

Because these three possibilities (<,>,=) are exhaustive, consistency implies the reverse, sometimes 
called "cancellativity" : 

= ly^z or z®xl = [z®y => a;i = ly (61) 



A. 2 One Type of Atom 

Consider a set of disjoint atoms {ai, a2, as, ... , a,., a^+i, . . . , ajv}, each of which is associated with the 
same value so that m(ai) = a for all i G [1,-^]. We will append such atoms one at a time, using the 
combination operator U to construct compound elements 

(((ai U as) U . . . U)a^) U a^+i (62) 

which are to be valued as 
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(((m(ai) ® m{&2)) ® . . .) © m{^r)) © m{^r+i) ■ (63) 
Since the atoms all have the same value, the subscripts are immaterial for valuation and we may write 

"1 of a" = ai, so that m(l of a) = m(ai) = m(a) = a (64) 

and 

"2 of a" = ai U a2, so that m(2 of a) — m(ai U a2) — m(a U a) (65) 
and so on with the addition of 

"0 of a" EE 0, so that m(0 of a) = m(0) = . (66) 
In principle, we could have any of 

positive style 

m(0 of a) <{ = I m(l of a) { nuU style (67) 



> 



negative style 



Null-style atoms all share the same value mg. If there were two such values, say mg and m^, then the 
equalities 

to(x) = mg ® to(x) = TOg © m(x) (68) 

for any x would, by cancellativity, make them equal. 

We proceed with atoms restricted to positive style, leaving the extension to negative (if required) 
until the end. Chaining a sequence of positive a's with another a yields, successively, the same nature of 
relationship between m(l of a) and to(2 of a), then m(2 of a) and m(3 of a), and by induction m(r of a) 
and m(r+l of a). Hence successive multiples are ranked by cardinality, and can continue indefinitely. 

m(0) < m(l of a) < to(2 of a) < • • • < m(r of a) < m(r+l of a) < . . . (69) 

Whatever values were initially proposed, we are free to regrade to other values of our choice, provided 
only that relevant order is preserved. Here, we are free to assign values as multiples 

m(r of a) = ra (70) 
of any positive value a > 0. The basic linear additive scale is now in place. 

Illustration 

We are not forced to adopt this linear scale, and a user's original assignments may well not have used 
it. We can allow other increasing series, such as m(r of a) = r^a, but we could not use a non-increasing 
series like m{r of a) = asin(r) without some values being the wrong way round. The only acceptable 
grades preserve order so that they can be monotonically reverted to the adopted integer scale (Figure 
5). 

A. 3 Induction to More Than One Type of Atom 

Suppose that sequences of atoms drawn from up to k types {a, . . . , c} are quantified as the grid of values 

fi(r, ■ ■ . ,t) = m{r of a and . . . and t of c) = ra + . . . + tc (71) 
^ ^ ^ ^ ^ 

multiples of up to k types in any order corresponding terms 

for positive multiples r, . . . ,t. Any individual marks that the atoms may possess beyond their type are 
ignored in this scalar representation. This hypothesis Equation 1711 is already the assignment for k — 1, 
and we aim to develop it to all k by induction. Before doing this, we note that commutativity is implicit 
in Equation [7T] for atoms or sequences drawn from the original k types, because 

^iir + r',...,t + t')^fl{r,...,t)+^l{r',...,t') (72) 
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But commutativity for /c > 1 is not being improperly assumed, because the inductive proof starts from 
k = 1, for which Equation [7T] reduces to the proven Equation [701 

We now append an extra type d of atom, and investigate values of the extended function 

fj.(r, . . . ,t;u) = m{r of a and . . . and i of c) ® m(u of d) (73) 

formed by appending, successively, u — 1,2,3,... new atoms. If a new value coincides with an 
already-assigned value, it is thereby determined. Otherwise, the new value must interleave (including 
lying beyond) existing ones, and we are free to assign it any convenient value within that particular 
interval (Figure 6). 

A. 3.1 Repetition Lemma 

To proceed, we need the repetition lemma, that if 

H{r,...,t){ = \^l{ro,...,to;u) (74) 



> 



then 



//(nr, . . . , ni) I = | ^(nrg, . . . , n^o ; ?^u) (75) 

for n-fold repetition. 

Suppose the lemma does hold for n. Prefix Equation [74] with "nro of a and . . . and nto of c" , and 
postfix with nu of d. 

fj,{nro+r, . . . ,Tito+t) (B ■m{nu oi d) 1^ = | ^((n+l)ro, . . . , (n+l)io ;(?t-+1)u) (76) 
Prefix Equation [7S] with "r of a and . . . and t of c" . 

/x((n+l)r, . . . , (n+l)t) I ^ | /x(nro+r, . . . , nto+i ; nw) (77) 



Because 



^{nro+r, . . . , nto+t) © 'm{nu of d) = ^(nro+r, . . . , nto+t ; nw) (78) 

(these two expressions being alternative notations for the same quantity), the relationships Equation 1771 
and Equation [75] combine to give 



Ai((n+l)r,...,(n+l)t) I = | ^((n+l)ro, (n+l)io ;(«+!)") 



(79) 



So, if Equation 1751 holds for n, it also holds for n + I. It does hold for n = I, proving by induction the 
repetition lemma for all n = 1, 2, 3, ... . 

A. 3. 2 Separation 

We define the relevant intervals for the new sequences /i(ro, . . . , to ! "") by listing the previous values 
Equation [7T] that lie below (set A), at (set B), and above (set C) the new targets (Figure 7). 



A 



< 



j: {r,...,t;u} such that ^(r, ...,t)| = | ^(ro, . . . , ; (80) 

This decomposition must hold consistently across all new sequences, for all u. Values for any particular 
target multiplicity u lie in subsets of A, B, C with u fixed appropriately. It is convenient to denote 
provenance with a suffix (1 for A, 2 for B, 3 for C), so that these definitions can be alternatively written 
as 



A 
B 
C 



{ri,...,ti such that ^(ri, . . . , ti) < ^(ro, . . . , io ; wi) 

{r2,...,t2;"2} such that /i(r2, . . . , ^2) = m(?'o, ■ • ■ , ; "2) (81) 

{r3,...,t3;u3} such that /Lf(r3, . . . , ia) > /u(ro, . . . , io ; M3) 
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Apply repetitions n = U2U3 for set A, and n = U1U3 for set B, and n = U1U2 for set C. 

A : IJ,{U2U3ri, . . .,U2U3ti) < n{U2U3ro, . . . ,U2U3to ■,UiU2U3) 

B : ^{uiU3r2,. ■ ■ ,uiU3t2) ^ ^i{uiU3rQ,. . . ,uiU3ta]UiU2U3) (82) 
C : n{uiU2r3, . . . , MlU2t3) > /i(M,l?i2ro, ■ ■ ■ ,'«iU2to ;ui'«2U3) 

Prefix various multiples of 'Vq of a and . . . and of c" . 



A 
B 
C 



fx{{uiU2 + UiU3)ro + U2U3ri, .. ., (W1W2 + UiU3)to + U2U3ti) < Q 

IJ,({uiU2 + U2U3)ro + Ul«3'"2, • • • , (^1^2 + U2U3)to + UiU3t2) = Q (83) 
H({U1U3 + U2U3)ro + UiU2r3, {uiU3 + U2U3)to + UiU2t3) > Q 



where 



Q = n{{uiU2 + U1U3 + U2«3)ro, . . . , {uiU2 + U1U3 + U2U3)to ; U1U2U3) (84) 
Evaluate the left-hand sides and eliminate the common right-hand sides Q. 

{{uiU2 + UiU3)ro + U2U3ri)a H h ((ztlM2 + ^1^3)^0 + U2U3ti)c 

< {{uiU2 + U2W3)''0 + UiU3r2)a H 1- {{uiU2 + U2U3)to + U\U3t2)c (85) 

< {{UIU3 + U2U3)ro + UiU2r3)a H h ((U1U3 + W2W3)to + UiU2t3)c 

Subtract (^1^2 + ^1^3 -|- U2U3){rQa H -|- toc) and divide by 'UiU2'U3- 

{{r\ - ro)a H \- {h - to)c) / m 



any member of A 

< {{r2 - ro)a H h (f2 - to)c) / U2 



any member of B 

< {{r3 - ro)a H \-{t3- io)c) / U3 

^ V ' 

any member of C 



(86) 



Taking {{1 — ro)a -!-••• + (* — to)c) /u as the statistic, all members of A lie beneath all members of 

B, which in turn lie beneath all members of C. We can now assign the value of /i(ro, . . . , to ; w) for some 
target multiple u. The treatment differs somewhat according to whether or not B is empty. 

A. 3. 3 Assignment When B Has Members 

If B is non-empty, we now show that all its members share a common value. Let two members be 
{r, . . . ,t;u} and {r', . ■ ■ ,t' ; u'} (the sufiix "2" is temporarily redundant), so that, by definition. 



H{r,...,t) = fi{ro,. . . ,to;u) 
li{r',...,t') = n{ro,...,to;u') 



Apply repetitions by u' and u, respectively. 



niu'r, . . . , u't) = /z(u'ro, . . . , u'to ; uu') 
^i{ur' , . . . , ut') = fJ,{uro, ■ ■ ■ , uto ; uu') 



Prefix multiples u and u' of 'Vq of a and . . . and to of c" 



li{urQ + ?i'r, . . . , uto + u't) = ^{ur^) + u'r^, . . . , wto + w'^o ; uu') 
li{u'rQ + ur' , . . . , u'tQ + ut') = iiiuro + u'ro, . . . , uto + u-'to ; uu') 



(87) 



(88) 



(89) 



Evaluate the left-hand sides and eliminate the common right-hand side. 

{urn + ur)a H h {uto + u't)c = {u'ro + ur')a ^ h {u'to + ut')c (90) 

Subtract {u + u'){rQa H + toc) and divide by uu' , 

(r - ro)a + ••• + (*- to)c ^ {r' - ro)a + ■ ■ ■ + {f - to)c ^ ^ 
u u' 

in which d denotes this common value now seen to be shared by all members of B. Using the definitions 
again, evaluating, and using this common value gives 
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n{rQ, . . . ,to;u) = fi{r, . . . ,t) — ra + ■ ■ ■ + tc — r^a + ■ ■ ■ + toc + ud (92) 

where d is seen to be the value m(d) of a single atom of type d. By Equation I?T1 this value is rationally 
related to the previous values a, . . . , c. 

Illustration 

Suppose for simplicity that only one type of atom has previously been assigned {k = 1), according to the 
integer scale m(r of a) — ra with a = 1. Suppose that the new atom d has value c? = |, rationally related 
to a. By 3-fold repetition, this means that m{i of d) lies exactly at 5, and is a member of set B. Again 
by 3-fold repetition, m{l of d) cannot lie at or below 1 because that would wrongly imply m(3 of d) < 3. 
Similarly, it cannot lie at or above 2 because that would imply to(3 of d) > 6. So m(l of d) necessarily 
lies between 1 (which lies in set A) and 2 (which lies in set C) and can without loss of generality be 
assigned |. Similarly, m(2 of d) necessarily lies between 3 and 4 and can without loss of generality be 
assigned and so on (Figure 8). 

These assignments obey axioms 1 and 2, and we now have a and d on the same linear scale. 



A. 3. 4 Assignment When B Has no Members 



When B is empty, the strict inequalities Equation [55] separating A and C imply that partitioning 
between them can be accomplished by some real 5. 

(ri-ro)aH h (ii - <o)c ^ c ^ ('"s - 7'o)a H ^ {h - to)c 

< d < (93) 

Ul U3 



any member of ^ any member of C 

For the target multiplicity u, the definitions Equation 1801 showed /i(r'o, . . . ,to ',u) to he bounded below 
by those members of A having ui = u, and bounded above by those members of C having — u. These 
constraints relevant to the target u are 

ria + ■ ■ ■ + tic < fj,{ro, . . . ,to ;u) < r^a + ■ ■ ■ + tsc (94) 

subset ui—u of ^ subset u^—u of C 

which is equivalent to 

(ri - ro)a-\ h (^i - to)c 



subset ui—u of j{ 



^ pt(ro, ■ . ■ ,to ;m) - (rpaH h tpc) ^^^^ 

u 

^ (ra - ro)a H h (^3 - to)c 

u 



subset u^—u of C 

Because this refers to subsets involving a single u rather than the entire sets involving all m, it is a weaker 
constraint than the preceding global constraint Equation|n3]was on 5. In other words, its central quantity 
{li{ro, . . . ,to;u) — (roo + • • • + toc)) / u is allowed to lie anywhere within an interval that contains the 
narrower interval containing d. Accordingly, it is legitimate to assign 

IJ,{ro,...,to;u) - (rpaH \- tpc) _ ^ ^^^^ 

u 

which automatically satisfies all the relevant constraints Equation 1951 So the simple assignment 

/i(ro,...,io;w) = ?'oaH htoc + U(5 (97) 

automatically falls in the correct interval. The only freedom is regrade to some alternative value within 
the relevant interval. 
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Illustration 



Suppose that three types of atom have previously been assigned (fc = 3), according to 

m(r of a) © m(s of b) ® m(t of c) = ra + sb + tc (98) 

with a = 1, 6 — ^y2, c — ^3. Now introduce a fourth type d. Omitting ro, sq, to for simphcity, we might 
find that muhiples it of d fall into successive intervals as follows. 

2.0000 = 2a < to(1 of d) < a + b= 2.4142 

4.4641 = a + 2c < m(2 of d) < 26 + c = 4.5605 

6.6569 = a + 46 < m(3 of d) < 5a + c = 6.7321 ^^^^ 

22.3424 = 14a + 6 + 4c < m(10 of d) < 9a + 76 + 2c 22.3636 

These are the constraints Eg nation [Ml relevant to each individual u = 1, 2, 3, . . . , 10, . . . . It is guaranteed 
that there exists some 6 such that the relevant interval for each target multiplicity u covers uS, as 
illustrated by the diagonal line of slope l/S in the diagram. Any breakout from these intervals would 
have contradicted axiom 2 thereby showing that S had been incorrectly assigned (Figure 9). 

According to Equation [53] with ro = sq = to = 0, the value of (5 = m(u of d) / u is constrained by all 
the members of A, B and C. 

By the time these sets have expanded to cover up to 10 copies of d, the surviving interval is 

2.2360 = (8a + 7c)/9 < 5 < (7a + 5c)/7 = 2.2372 (100) 

from ui— 9 from U3— 7 

and by the time 1000 copies are allowed, the union of all the constraints fixes 5 to 10 decimal places. 

2.236067977497 = 1345a+566+359c ^ ^ ^ 80a+5456+286c ^ ^ 236067977505 (101) 

915 602 

^ V ' " v ' 

from ui— 915 from 113— 602 

(The example happened to have 5 = -\/5.) 



Accuracy 

The gap between A and C might allow 5 to be uncertain. We assume that 5 is bounded below, otherwise 
the appended atoms of type d never have measurable effect. This implies the existence of u such that 
u5 > na for any multiple n, no matter how large. We also assume that S is bounded above, otherwise 
even a single d atom always overwhelms everything else. This implies the existence of a greatest r > n 
such that ra < uS for that u. Taking other types of atom to be absent for simplicity, we have 



{r,0,...,0;u) £A and (r + 1, 0, . . . , ; u) G C (102) 

where r can be indefinitely large. The corresponding inequalities ra/u < S < {r + l)a/u from (|93p fix S 
to accuracy 1 part in r (1 in n or better). 

This proves that 6 can be found to arbitrarily high accuracy by allowing sufficiently high multiples. 
Denote the limiting value of S by d. This value TO(d) = d of a single atom of type d is now fixed to 
unlimited accuracy, but has no rational relationship to the previous values a, . . . , c. 



A. 3. 5 End of Inductive Proof 



Whether or not B had members, the assignment 



^(ro, . . . , to ; "") = tqu + ■ ■ ■ + toc + ud (103) 

obeys all the defining inequalities Equation 1801 This updates the original assignment Equation [7T] from 
k atom types to fc + 1, so by induction from fc = 1 it holds for any k. 

m{r of a and . . . and t of c and . . . and w of e) = ra + ■ ■ ■ + tc + ■ ■ ■ + ve (104) 

^ V ' " V ' 

any number of types in any order corresponding terms 
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Atom types in the above expression are often different, but do not need to be, and the formula 
represents the quantification of a general sequence. Embedded in it, and equivalent to it, is the sum 
rule X (B y = X + y for the values m(x) = x and TO(y) = y oi arbitrary sequences. Any order-preserving 
regrade Q is also permitted, but no order-breaking transform is permitted. 

This completes the inductive proof for atoms of positive style. The proof holds equally well for 
atoms of negative style, for which the values are negative. Meanwhile, Equation [BS] shows that atoms 
of null style have zero value. So, even if the atoms may have arbitrary style, Equation 11041 offers the 
only consistent combination rule. The result thus holds for atom values of arbitrary sign and arbitrary 
magnitude, though the nature of the constructive proof requires atom multiplicities to be non-negative. 
□ 

A. 4 Axioms are Minimal 
Theorem: 

Axioms la, lb, 2 are individually required. 



Proof: 

We construct operators O ( "not quite 0" ) which deny each axiom in turn, while not being a monotonic 
strictly increasing regrade of addition. 

Without axiom la (postfix ordering), the definition 



aOb=[a\+b (105) 

where [aj is the integer at or immediately below a, satisfies axioms lb and 2 but cannot be equivalent 
to addition because it is not commutative; aOb ^ bO a. So axiom la is required. 

Without axiom lb (prefix ordering), the definition 



aOb = a+[b\ (106) 

satisfies axioms la and 2, but cannot be equivalent to addition because it is not commutative. So 
axiom lb is required. 

Without axiom 2 (associativity), the definition 

xOy = x'^ +y'^ (107) 

satisfies axioms la and lb (ordering), and also happens to be continuous and commutative 
(xOy — yOx). Yet it cannot be equivalent to addition because <d{xOy) = &{x) + &{y) has no 
solution that would enable a regrade 8. That can be shown by appropriately differencing SxSy to reach 
&{z -I- e) — 2Q{z) + &{z — e) = whose solution 9(z) = Az + B fails to satisfy the supposedly defining 
Eouation 11071 Hence ordering is insufficient even when accompanied by continuity and commutativity. 
Axiom 2 (associativity) is definitely required. □ 



B Appendix B: Product Theorem 

Theorem: 

The solution of the functional product Equation 

^{T + + ^{r + v) = '^{r + C{^,v)) (108) 
in which r, ^ and t] are independent real variables and ^ is positive is 

^'(a;) = Ce^^ (109) 
where A and C are constants (C necessarily being positive). 
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B.l Proof: 



The quoted solution is easily seen to satisfy the product equation, which demonstrates existence. The 
remaining question is whether the solution is unique. 

First, we take the special case ^ = 77, so that C ~ C and C — ^ take a common value a. This gives a 
2-term recurrence 



2*(T + C-a) = *(t + C) (110) 

in which r and C remain independent, though a might be constant. In fact, a must be constant, otherwise 
there would be no solution for ^. Consequently, ^ behaves geometrically with 

*(6» + no) = 2"*(6I) (111) 

for any integer n, 6 being arbitrary. Although this plausibly suggests that ^ will be exponential, that is 

not yet proved because could still be arbitrary within any assignment range of width a. 

To complete the proof, take a second special case where C — ^ and {( — rj) / 2 take a common value b. 
This gives a 3-term recurrence 

*(r + C-fo) + *(r + C-2fe) = *(r + C) (112) 

in which r and ^ remain independent, though b might be constant. In fact, b must be constant, otherwise 
there would be no solution for ^. The solution is 

for any integer m, 9 being arbitrary. 

This combines with the 2-term formula to make 



(114) 



For any integer n, there is an even integer m for which < mb — na < 2b so that all three arguments of 
lie in the range [9, 9+2b]. As n is allowed to increase indefinitely, so does this m in proportion m/n ~ a/b. 
Depending on the sign of n, at least one of the exponents ±m log ^-^y^ ~ ^"-"S ^ ^^"^ become indefinitely 
large and positive. Unbounded values of ^ being unacceptable, the coefficient of that exponent must 
vanish. So either 



^{9 + mb- na) = ^{9) e"^°s (^)-"i°s2 ^^^g^ 



(first term only) or 



^{9 + mb-na) = (-1)™^'(0) e-™i°s (ne) 
(second term only, and even m makes the sign (—1)™ = 1). In the first case, bounded * requires 

b log(i^) 



a log 2 

and in the second case, bounded * requires 



(117) 



a log 2 

Either way, 



'--MM) 



^{9 + mb- na) = ^{9) e^^'"""""^ (119) 

with A constant. 

Although this strongly suggests that will be exponential, that is not yet fully proved because offsets 
mb — na with even m are only a subset of the reals. There could be one scaling for arguments 9 of the 
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form mb — na, another for the form \/2 + nib — na, yet another for tt + mb — na, and so on. Fortunately, 
b/a is irrational, so the offset mb — na can approach any real value x arbitrarily closely. Express x as 
X — mb — na + e with m and n chosen to make e arbitrarily small. Then 

m(x) = e^("^-"°%(e) = e'^(^^^)*(e) « e'^^*(e) (120) 
because C^*^ ~ 1. Separating variables, ^'(e) ~ constant, giving 



(solution) ^{x) = Ce-^^ (121) 

to arbitrarily high precision (e — >■ 0) with constant C. 

This obeys the original product equation without further restriction and is the general solution, with 
corollary e^^ + e^^ — e^^ defining CHiV) ^-nd confirming that a = yl^^log2 and b = log(i^i^^) 
were appropriate constants. □ 

The sought inverse, in terms of the constants A and C, is 

(inverse) e(M) = ^log^ (;L22) 

in which u and hence C are both positive. 



C Appendix C: Variational Theorem 

Theorem: 

The solution of the functional variational equation 

U'lm^my) = X{mx) + /i(rny) (123) 

with positive mx and my is 

H{ni) = A + Bm + C{m\ogm-m) (124) 

where A, B, C are constants. 
C.l Proof: 

The quoted solution is easily seen to satisfy the variational equation, with corollaries that the functions 
A and /i are logarithmic, which demonstrates existence. The remaining question is whether the solution 
is unique. 

Write log — u, logmj, = w, and rewrite the functions as A*(u), n*{v) and H'[m) = hilogm). 

h{u + v) = X* {u) + {v) (125) 
Put u = to get \*{u) ~ h{u) — constant and u = to get /^*(w) = h{v) — constant. 

h{u + v) ^ h{u) + h{v) ~ B (126) 
This is Cauchy's functional equation ([13]) 

f{u + v) = ![u) + f{v) (127) 

for /(t) = h{t) — B from which f{nt) — nf{t) and then f{-^t) — ^/(i) follow by induction for integer r 
and n. Hence 

fit) = ct (128) 

where c = f{to) /to evaluated at any convenient base tQ. Awkwardly, the recurrence only relates to 
a rational grid — there could be one value of c for rational multiples of 1, another value for rational 
multiples of y/2, yet another for rational multiples of tt, and so on. Fortunately, the sought function H 
is an integral of /, on which such infinitesimal detail has no effect. 
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To show that, we blur functions (t>{u,v) by convolving them with the following unit-mass ellipse, 
chosen to blur u, v and u+v equally, according to 

v)=jjdxdy ^ ' "^^^ - ^' ^ - y) (129) 

For small width e, blurring has negligible macroscopic effect. The convolution transforms the Cauchy 
equation to the same form 

F{u + v)= F{u) + F{v) (130) 

as before, with the new function 

Fit) = r dx ^XEIZ fit - x) (131) 

being a continuous version of the original /, narrowly blurred over finite support. With continuity in 
place, the Cauchy solution 

F{t) = Ct (132) 

can only have one value for the constant C. 

Finally, the definition dH/dm = /i(logm) = B + /(logm) yields 

/m 
/ (log m' ) dm' (integrate) 

/log m 
f{t) e*dt (change variable) 

dx ^ / f{t) e^dt (insert blurring) 

-e Tre^ J 

/« 2,/(r2 _ ^2 rx+logm 
dx / f{t-x)e*-''dt (offset dummy t) (133) 

/e 2\/e'^ — x^ /-log™ 
dx — 2 J /(* - ^) {\x\ < e small) ! 

/■logm 

= Bm + 



/logm 
F{t) e*dt (definition of F) 

/log m 
te^dt (substitute) 

Hence, to arbitrarily high precision (e ^ 0), H integrates to 



/"log m 

Bm + C 



H{m) = A + Em + C{m. log m. - m) . (134) 

This obeys the original variational equation with corollaries A(a;) = Bi + Clog(a;) and iJ,{x) = B2 + 
Clog(a;) where Bi + B2 = B, and is the general solution. □ 
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Tabic 1: Operators and their symbols. 



Operation 


Symbol 


Quantification 


(Eventual form) 


ordering 


< 


< 




combination 


U 


e 


(addition) 


direct product 


X 


o 


(multiplication) 


chaining 







(multiplication) 



Figure 4. Cartoon graphic of the symmetries invoked, and where they lead. Ordering is drawn 

as upward arrows. 



Fidelity 



0: Values increase 



Ik 



Positivity 



Combination 



1: U preserves order 



2: U is associative 



t ^ t and t 
(*U(J')U* = *U(CJ'U*) 



^ ' — 



Measure 



Direct product ^i^^c^p (^i^^c^p) x t] 

3: X is distributive Jj, c;p ^ ♦ x l| ^ x [] 



4: X is associative (♦ x 1^) x K = J|k x (t] x H) 



Divergence 



Implication 










17} 










5: chaining is associative 




= 










t J 


t" } 






* 



Measure — > Probability and Bayes 
Divergence — > Information and Entropy 
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Figure 5. Ordered multiples can be placed on an integer scale, here drawn with a = 1. 



a 2a 3a 4a 5a 6a 7a 




1 2 3 4 5 6 



Figure 6. A new value, displaced away from the existing grid, immt lie within some interval. 
Any assignment oiitsidc the strict interior would be wrongly ordered, while any value inside could 
be reverted to some other selection by order-preserving regrade. 



old grid value 

/x(ro,...,to)^^ 

' " ^ 

• • • • • 


_ value after u of d appended 
assignment of new 
' 

o 


• • • • 


1 M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 

iiit(>r\al " 



Figure 7. The interval encompassing the new value lies above set A and below set C. 



previous grid values 
• • • • • 


V ' 

O 


previous grid values 
• • • • 


1 M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 1 M M 

^ " ' " iuLerval ^ ^ " ' 

set A set C 



Figure 8. ]\lultii)les of a iivw of atom can l)c asriigiicd linear \alues. 



3d 



ji 1 1 1 1 1 



1 1 1 1 1 1 1 1 1, ii 1 1 1 1 1 1 1 1, ii 1 1 1 1 1 1 1 1, ii 1 1 1 1 1 1 1 1, J 1 1 1 1 1 1 1 1, ii 1 1 1 1 1 1 1 1, ii 1 1 1 1 1 1 1 1, ii 1 1 1 1 1 
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Figure 9. Multiples of a new atom can always be assigned linear values 6, 26, 36, An 

individual multiple can be assigned anywhere within the corresponding interval, but the linear 
assignment can always be chosen. 
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